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ABSTRACT 

• u predictive validities of criteria for entrance 

into a teacher education program were studied, based on fhe 
hypothesis that criterion specificity is positively reUte3 to the 
predictive validity of the selection measure. SuSjJc" (N=512) veJe 

filS LcLL2 ;i J* ^^«f"^» «tate university. Measures from student 
files included the following predictor variables: (l) undergraduate 

Ed^cfrf''" c^yf** ^""^""V (2>.«o"« o« the California Basic 
Educational Skills Test (CBEST) ; (3) scores on the National Teachers 
Examinations; and (4) prerequisite education course grad« T^re^ 
composite criterion scores of performance were S^Jed''^ sJudl^J 
reluct llllZT'u " university superv^o^s an" 

enablef ti! • f: °''-'^*"*te correlations and multiple regression 
enabled the simultaneous consideration of all predictors and theiJ 
Ztll7" ^"'P*''-'^""' Consistent with the study's hy^oSesisf Hi 
methods courses were superior in their ability to accou" f;r 
TLn"?" 'l^t'''* criterion measures. Overall. h?A was aUo 

eSs ^:U sLrr:i''-'' f •''^ 'l'''''^' ^"^ t"^* standardized 

tests fell short of significance in 10 of 12 trials T«i« i ; ^- • ^ 

selection of teacher candi&s are discussed el J!ki1! I'*'' 
«fiiHv « ^ 1.1- , aiscussea* Three tables present 

study data, and there is ajff-item list of references. (SLD) 
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INTRODUCTION 

The validity of admission criteria utilized for entrance 
into a teacher education program is vitally important. Present 
national and state policies have called for higher and higher 
admission standards in response to reform reports calling for 
higher quality teachers. These standards often take the form of 
higher grade point averages or some form of standardized tests. 
The U.S. Department of Education reports that more than 20 states 
now mancate some form of testing for candidates to be admitted 
into teacher education programs (Office of Education Research and 
Improvement, 1987, p. 14). 

While a variety of criteria may, be used for admission to 
teacher education programs, little is known about the true 
objective value of these measures in reference to the selection 
of higher quality candidates. m many systems, selection 
practices appear based on convenience of predictors available 
and/or ■common sense' decisions about what measures "should- 
predict good teaching performance. Little research has been 
conducted to support the predictive validity of newly implemented 
selection measures. 

Validity does not refer to the selection criteria score or 
test itself, but to the quality of the inferences or decisions 
based upon the selection test score (APA, 1977; APA, 1985). The 
selection process is not valid if it does not result in the 
selection of those students who will make the best teacher.?. 
Unfortunately, supported by a misguided court system, content 
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validation procedures have often been substituted for methods 
which actually assess the ability of the selection scores to 
enable predictions relevant to future performance (Rosenfeld, 
Thornton, & Skurnik, 1386). Content validation is a process more 
suitable for making inferences about the quality of the process 
of initial test construction. Content validation is NOT suitable 
for making inferences about the meaning of the selection score or 
its ability to enhance the decision process (Tenopyr, 1977; SIOP, 
1987). If the purpose is to infer how well a candidate will 
perform on the job, we must determine the mathematical 
relationship between test scores and some numerical index of job 
success (Lawshe, 1985; SIOP, 1967). -This requires criterion 
validation. Anything less is described as unethical by division 
14 of the APA (SIOP, 1987), 

Nevertheless, most programs continue to use selection 
criteria that have never been subjected to the appropriate 
validation assessment. The use of Invalidated admission 
standards may result in serious consequences, not only for the 
potential teacher education students, but also for the districts 
in need of a teaching force. If the utilized criteria have no 
relationship to actual teaching performance, potentially 
excellent teachers are prohibited from joining the profession. 

The use of invalid standardized tests as selection hurdles 
has special implications for ethnic minorities* When potential 
teachers from diverse populations are excluded from entrance into 
teacher education programs on the basis of standardized test 
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scores, school districts are denied the opportunity to increase 
the diversity of their teaching force — an unnecessary and 
intolerable outcome. Cooper and Williams (19S6) identified this 
disturbing trend in states that required competency testing for 
certification. The proportion of Black teachers within the 
teaching force decreased the longer the mandated testing had been 
in place. No similar trend was reported in states without 
mandated testing (Cooper & Williams, 1986). 

The California State University system incorporated stricter 
standards in the form of higher GPA's, successful completion of 
the California Basic Educational Skills Test (CBEST), and passage 
of the National Teachers Exam (NTE) or an approved waiver 
program. Little research, however, has been done to examine the 
relationship between these scores and measures of subsequent 
student success in the teacher education program* 

The few, rather limited studies available in the general 
literature that have attempted to investigate this question of 
predictability have provided mixed results. In two studies 
conducted by Olstad (1983), GPA was identified as a predictor cf 
relative success in student teaching performance, while 
California Achievement Test scores were not supported as 
selection criteria. 

Neither the NTE Test of Professional Knowledge nor student 
GPA predicted student teacher performance as measured by 
cooperating teachers in a study conducted by Dobry, Murphy, and 
Schmidt (1985). 
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Selection Criteria - 5 
Mayer* 3 !;i3c9) study of admission requirements reported nc 
significant relationsnips between students exceptionally admitted 
to the teacher education program (due to low GPA ) and time spenT: 
by university supervisors. University supervisors' evaluations, 
students' self-report of preparation time, completion of program, 
and teaching credential acquisition were also not significantly 
related t.o the predictor. However, classroom teachers* 
assessments of exceptional admits were found to be significantly 
lower than those of regularly admitted students. Results 
regarding specific predictors were unavailable due to the 
multiple categories used to define exceptional admits, 

A preliminary study by Riggs and Riggs ( 1 991 )' supported the 
use of previous academic achievement, both in the form of overall 
GPA and grades earned in early teaching methods courses. 
Standardized test scores received little support. 

The question of admission standards is critical to reform 
efforts. Given the mixed results of the Vmited studies 
addressing this issue, the current project attempted to provide 
further evidence toward the resolution of this important 
question. Improvements in the current study over previous 
investigations include a much larger sample and a more reliable 
performance indicator based on multiple sources of evaluation. 
HypQthesi s 

The hypothesis for this study is based both on results of 
previous research and the general principle that criterion 
specificity will be positively related to the predictive validity 
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of the selection measure. In the continuum of criterion 
specificity, grades from educational methods courses are based on 
developed abilities i^osz exclusively related to the performance 
of tasks required by a teacher. The overall GPA is less 
specific, but nevertheless reflects achievement in a course of 
study designed for future educators. Broadly-oriented 
achievement tests designed to assess minimum competency levels of 
general developed abilities are ohe least specific. 
Consequently, it is predicted that methods course grades will be 
the best predictors of success in student teaching. GPA is 
expected to be less predictive, but still significantly 
correlated with the performance measure. The standard i ::ed 
achievement tests (CBEST subscales and WTE ) are nOT: expected to 
account for significant variance in the student teaching 
assessments . 

METHOD 

Subj ects 

Subjects (n = 512) were drawn from a population of 
approximately 1200 students admitted to the teacher education 
program of a small California state university. 
M easures 

Measures obtained from student files included the following 
predictor variables: 

1 ) undergraduate GPA, 

2) CBEST scores, 

3) NTE scores, 

4) prerequisite education course grades. 

The prerequisite education courses were in educational 
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psychology, reading methodology, and math methodology. All three 
courses were taken prior to admission to student teaching. 

Three composite criterion scores were based upcn s-::udent 
teaching performance as measured by university supervisors ana 
resident teachers. University assessment forms were completed by 
supervisors and resident teachers at the midpoint and at the end 
of each of two ten week student teaching blocks. University 
supervisors based their evaluations upon a minimum of five 
visitations, while resident teachers based theirs upon daily 
observations. A mailing requested additional evaluation of 
student teaching performance by supervisors and resident 
teachers. 

Both parties assessed student teacher competencies as they 
are presented within the university's Student Teacher Evaluation 
Form. These competencies were described in depth within a 
Student Teaching Handbook that all supervisors, cooperating 
teachers, and student teachers utilized. 

The five categories assessed within the Student Teacher 
Evaluation Form were: planning, instruction, evaluation, 
classroom organization, and classroom behavior (discipline). Ml 
categories were assessed for both individualized/small group 
instruction and large group instruction. Each category was rated 
with both of the following five-point scales: 

Opportunities Taken 

1 Nearly all opportunities to demonstrate progress 
taken . 

2 Most opportunities to demonstrate progress taken 

3 Some opportunities to demonstrate progress taken.' 
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4 Few opportunities to demonstrate progress taken. 

5 Opportunities to demonstrate progress seldom taken. 

Attainment Demonstrated 

1 Functions independently on most goals at level cf fully 
qual if ied teacher. 

2 Functions independently on several goals with 
occasional supervision needed. 

3 Functions adequately on several goals under 
supervision. 

4 Has difficulty attaining several goals even under 
supervision. 

5 Has been unable to attain most of the goals. 

An additional appraisal form mailed to both supervisors and 
res-dent teachers assessed the five performance components, but 
used a different response format. 

The three final composite criterion measures were computed 
by collapsing multiple evaluations obtained from: 1) resident 
teachers, 2) university supervisors, and 3) a combination of 
both. Scores were converted to percentages of total possible 
points. Combination of the measures was justified by their high 
intercorrelations, especially within the first two groups 
(Dunnette, 1963). This combination is also supported by the fact 
that performance data based on multiple observations are mo.re 
reliable/stable than those based on a single report. 
Anal vsis 

With the exception of Riggs and Riggs (1991), previous 
investigations categorized either their predictor and/or their 
criterion measures and tested criterion validity with either Chi- 
square analysis or analysis of variance. For example, student 
teaching performance rankings were trichotomi zed into "high, 
medium, and low" scores to enable analysis of variance on group 
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scores. This procedure reduces the power of such Ti^r.^*;" 
because meaningful variance between members within eac = 
lost , 

The present study employed more appropriate methca^ 
analyses that enabled the retention of the full range of ■ i : 
in all continuous measures. In addition to univariaLe 
correlations, multiple regression was used to enable tf^e 
simultaneous consideration of all predictors and to asjie.^-- .? 
relative i mportance . of each. 

RESULTS 

Descriptive statistics for all relevant variables .^rc- 
reported in Table 1. with the exception of method" cour:.e .jmjijs 
(which were skewed due to a very disproportionate number ..f -'j-. 
all distributions were approximately normal. The skew ana 
accompanying reduction of variance in the methods course rt.^ixSijt -^s 
reduces their potential to "covary" with the performance 
measures. This does not rule out their potential to perfcr^i u3 
significant predictors, but it does limit the potential magni t..de 
of the correlation-based statistics. 

Pearson correlation coefficients between the eight potential 
predictor variables and the three composite criterion measures 
are reported in Table 2. As pairwise deletion of missing data 
was used, n's and exact p's are reported for each correlation. 
The predictor variables are ordered in reference to their rank ^n 
the magnitude of their average correlations with the criterion 
measures. A full correlation matrix which includes the 
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correlations within predictor and criterion groups is Hncluded in 
Appendix A. 

Consistent with this study's hypothesis, the methods courses 
proved to be superior in their ability to account for variance in 
the criterion measures. The three courses correlated 
significantly with the three performance measures 12 of 12 
trials. correlations ranged from r = .10, g = .034 to r = .20, o 
= .000. 

AS predicted, overall GPA was also significantly correlated 
with all three criteria. The absolute magnitude of the 
correlations between GPA and teaching performance ranged between 
I = .14. fi = .04 to r = .15, o = .03. The correlations were 
slightly less, on the averc.ge, than those produced by the methoas 
courses, but the differences failed to reach statistics! 
significance. 

The standardized tests fell short of significance 10 .-Df 12 
trials. The CBEST reading and writing subscales produced 
statistically significant correlations of r = .12, p = .01 and r 
= .09, B = .035 with the performance evaluations submitted by the 
resident teachers. The maximum correlation resulting with anv 
pairing between the nTE core battery and the teaching perf or,...nce 
measures was r = -.029. All correlations with the NTE. though 
near zero, were negative in direc.ion. 

Stepwise multiple regression was also utilized to assess the 
potential predictive validity of selection test scores in optimal 
combinations. The NTE was omitted from these analyses, both 
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because it significantly reduced the viable sample size and due 
to its apparent impotency as a predictor as evidenced by tne 
univariate correlations. The results of the multiple regressions 
are reported in Table 3. All of these analyses further support 
the superiority of the methods courses as predictors of student 
teacher success. 

When predictors were regressed on the performance criterion 
produced by the resident teachers, the reading methods course 
(£0-345) entered the equation first. This variaole accounted for 
^% of the variance in performance. No other variables 
significantly increased R. 

When performance as reported by the university supervisor 
was the criterion, the math methods course (ED-34.0) entered 
first, accounting for 3% of the variance. ED-345 entered second, 
significantly increasing the R to .22 and the to 55^. 

Using the total performance composite, the reading methods 
course once again entered first producing an R of .21. The 
educational psychology course entered second, swelling R to .2*1 
and accounting for 6% of the variance in the performance measure. 

DISCUSSION 

Though the validity coefficients produced in this study were 
not of the magnitude that one might hope, they are of average 
size. Boehm's (1982) review of published criterion-related 
validity analyses reported that the average of validity 
coefficients from 176 studies was r = .219. The coefficients in 
this study were probably afflicted by maladies common to most 
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validation studies. The performance criteria were less than 
perfectly reliable and restricted in range (students scoring 
lowest in the predictor measures were not admitted to the teacher 
education program), and the most effective predictors were badly 
skewed. Nevertheless, the variance accounted for was not so 
small as to be of no practical importance. In certain selection 
circumstances, even a validity coefficient of ,20 can 
significantly improve the "hif rate in selection decisions (Saal 
3c Knight, 1983) . 

It is reassuring to find the positive relationship between 
methods course grades and student teaching performance. Courses 
which teach pedagogy should relate to ultimate teaching 
performance. This finding suggests we might place more Thrust in 
our own ability as teacher educators to evaluate our students' 
ability to teach within the education course setting, thus 
producing data that can be used as a major selection criterion. 
Certainly, at the very least, prerequisite methods course grades 
should be given the same consideration as other selection 
variables . 

Consideration of students' grades within methods type 
courses might prove especially helpful in the case of low overall 
GPA candidates. Present practice often "exceptionally" admits 
low GPA students to education programs or student teaching. when 
possible, final admission might be postponed for these students 
until they have had a chance to "prove" themselves within their 
methods classes. This study's results suggest that even a B 
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within such a class may be indicative of potential weakness in 
student teaching performance. Yet, the low GPA candidate who 
excels within methods coursework might be considered a worthwhile 
risk. 

Selection committees might also consider collecting more 
selection data from the methods instructors in aaaition to the 
course grade. This is especially important given the skewed, 
restricted variance reflected in most class grade distributions. 
The Pre-teacher Assessment (Gerlach ^ Millward, 1989) was 
developed to collect and utilize observational field data within 
a methods course. The authors do not recommend its use for 
selection, but believe it facilitates undergraduate preservice 
teachers* development of teaching skills. Perhaps, though, this 
assessment or similar others could be used to help make selection 
decisions, especially when considering students with low GPA or 
low standardized test scores. 

When selection practices are based on multiple objective 
criteria, the resulting selection decisions should be more valid 
and possibly more equitable. It has been demonstrated that 
under^-represented student groups may be disadvantaged by many 
standardized tests (Cooper & Williams, 1986). While policy 
calls for a more diverse teaching force, current practice may 
lock out many potential teaching candidates. The use of 
performance--based evaluation in early methods courses in addition 
to other selection measures, would enable such candidates to 
better demonstrate their teaching potential. 
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Finally, the worth of selection criteria must oe 
demonstrated to the public. National reports have raisec ^r.arv/ 
questions concerning the competence level of existing teac^^ers 
and the teacher education programs which produced therr. 
Validation studies must go beyond investigation cf the 
relationship of predictor variables to student teacher 
performance, Preservice teachers' performance must be studied as 
they complete their first years of teaching within their own 
classrooms. Only then will teacher education programs be 
accountable to the public for the teachers they prepare. 
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Table 1: Descriptive statistics for Predictor and Criterion 



Variables 

Variable _n Mean Std Dv 

Predi ctors : 

GPA 486 3,11 .45 

CBEST - Reading. 492 54.52 10.54 

CBEST - Math 492 54,80 11.25 

CBEST ^ Writing 492 46.31 7.91 

Education Psychology 437 4.08 1.20 

Math Methods 448 4.43 ,89 

Reading Methods 485 4.14 1.11 

NTE Core Battery 189 667,80 ' 6,64 
Cri ter ia 

Resident Teacher Eval 375 .89 .11 

Supervisor Eval 403 ,89 ,11 

Combined Evals 420 .88 .10 
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Table 2: Pearson Correlation coefficients between Selection 
Predictors and Performance Criteria 

Cr i ter i a 

Resident University Combined 
Teacher Superv i so r Evals 

Predi ctors 



Reading Methods 


P. 


.2000 
(364) 
= .000 




.1623 
(330) 
= .001 


P. 


. 1967 
(^07 ) 
=. .OCO 


Educat-ional Psychology 


a 


.1343 
(363) 
= .005 




. 1523 
(389) 
= .001 


Q 


. 1342 
( 406 "; 
= .000 


Math Methods 


Q. 


. 1 001 
(335) 
= .034 


B. 


.17 74 
(363) 
= .000 


2 


1 c o c 

37 3 ) 
= .001 


GPA 




. 1457 
(355) 
= .003 


D 


. 136S 
(383 ) 
= .004 


Q. 


.1433 
(399 ) 
= .002 


CBEST - Reading 




. 1 182 
(366) 
= .012 


E 


.0564 

(:.92) 


2 


.0652 
(409 ) 
= .094 


CBEST - Math 




.0598 
(366) 
= .127 


fi 


. 034S 
(392 ) 
= .2/16 


Q. 


.0421 
(409 ) 
= . 1 98 


CBEST - Writing 




.0945 
(366) 
= .035 


& 


.0055 
(392) 
= .457 


B 


.0289 
(409) 
= .280 


NTE 


e 


.0293 
(146) 
= .363 


B 


-.0224 
(158) 
= . 390 


B 


■ .0050 
( 165 ) 
= .^15 



Coef f icient/(Cases)/1-tai led Significance 
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Table 3: Regression Analyses 



Anal ysi s 


#1 : Dependent Measure - 


Resident T 


eacher Eval uation 




Variable ■ R 


R Sq 


Beta T 


Si g T 


Step 1 


Reading Methods .20 


.04 


.20 3.73 


.0002 


Anal ysi s 


#2: Dependent Measure - 


Suoervi sor 


Eval uation 






Variable R 


R So 


Beta T 


Siq T 


Step 1 


Math Methods . 18 


.03 


.18 3.43 


.0007 


Step 2 


Math Methods .22 
Reading Methods 


.05 


.15 2. 74 
.13 2.36 


.006^ 
.0135 



Anajj^si_s_»3_: D ependent Measure - Comb i n ed_Ey^uat ion 





Variable 


R 


» R Sq 


Beta 


-T- 
1 


Siq T 


Step 1 


Reading Methods 


.20 


.04 


.20 


2 . 90 


.0001 


Step 2 


Reading Methods 


.24 


,06 


. ^6 


3 . 05 


.0025 




Educational Psych 






.14 


2.72 


.0069 
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